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ABSTRACT 

The  U.  S.  Navy  currently  uses  the  Aviation  Selection  Test  Battery  (ASTB)  to  select 
pilots  and  naval  flight  officer  candidates  prior  to  their  entry  into  basic  flight  training.  The 
ASTB  began  operational  use  in  1992,  and  consists  of  5  scored  paper-and-pencil  subtests 
that  are  converted  to  composite  scores  that  are  predictive  of  training  performance  and 
attrition.  One  of  these  subtests  is  the  Biographical  Inventory  (BI).  The  BI  is  a  series  of 
questions  on  an  individual’s  background  and  life  experiences,  and  it  was  originally 
validated  to  predict  early  flight  training  attritions.  Factor  analytical  studies  have  shown 
the  71  item  BI  to  measure  traits  such  as  Risk-Taking  Behavior,  Athletic  Orientation, 
Military  Interest,  Engineering  Background  and  Independence.  Since  1992,  an  unscored 
bank  of  non-cognitive  test  items,  called  the  Aviation  Interest  (AI)  test,  has  been 
administered  to  all  applicants  to  the  Navy’s  flight  training  program.  The  current  study 
evaluated  the  factor  structure  of  this  unscored  subtest  with  the  current  BI  and  the 
incremental  validity  the  AI  items  might  provide  using  a  dichotomous  pass/fail  criterion. 
The  factor  analysis  showed  a  large  amount  of  overlap  between  the  items  in  the  AI  and 
BI.  However,  at  least  one  factor,  labeled  rebelliousness,  was  unique  to  the  AI  test.  Using 
a  horizontal  weighting  method  for  individual  test  item  responses,  a  combined  AI/BI  test 
key  is  proposed  to  improve  the  current  predictive  validity  of  the  ASTB  (R  =  .096)  and/or 
replace  outdated  BI  items. 

INTRODUCTION 

Attrition  in  the  training  of  naval  aviators  is  a  continual  concern  for  the  U.  S.  Navy  for 
both  financial  and  operational  reasons.  The  least  expensive  training  pipeline  is  the 
helicopter  syllabus,  in  which  a  late  stage  attrition  of  a  student  represents  a  cost  of 
approximately  $500,000.00.  The  more  expensive  training  pipelines  such  as  Jet/Strike 
pilot  training  cost  well  over  1  million  dollars  for  each  failure.  In  addition  to  the  financial 
costs,  failures  during  training  ultimately  impact  the  manpower  levels  in  the  fleet,  which 
can  negatively  impact  operational  readiness.  The  reasons  for  attrition  from  flight  training 
are  varied.  A  certain  percentage  of  prospective  aviators  are  dropped  from  the  training 
program  for  medical  reasons.  However,  the  majority  of  the  attritions  out  of  training  are 
due  to  academic  or  flight  performance  failures  or  to  individuals  just  choosing  to  leave 
the  program,  referred  to  as  a  Drop  On  Request  (DOR).  DORs  include  individuals  who 
have  chosen  to  leave  the  program  due  to  motivational  problems  or  an  expressed  lack  of 
interest  in  the  flight  program.  The  overall  attrition  rate  in  U.  S.  Navy  pilot  training 
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fluctuates  on  an  annual  basis,  but  for  the  last  20  years  or  so  the  rate  has  stayed  between 
15  -  25%  of  the  entering  student  population.  The  majority  of  these  failures  occur  during 
the  first  100  hours  of  flight  training. 

One  means  to  reduce  attritions  or  maintain  an  acceptable  level  of  training  failures  is  via 
ab  initio  selection  tests  validated  to  predict  performance  and  non-performance  based 
failures.  The  Navy  has  used  a  paper-and-pencil  test  to  predict  performance  during  initial 
flight  training  since  World  War  II.  The  first  test  battery  developed,  the  Aviation 
Classification  Test,  included  both  cognitive/spatial  abilities  tests  and  a  biographical 
inventory  which  focused  on  past  life  experiences  and  an  individual’s  personal  interests. 
Subsequent  versions  of  the  test  in  1952,  1971,  and  1992  retained  these  measures  as 
predictors  of  performance.  The  current  version  of  the  test,  the  Aviation  Selection  Test 
Battery  (ASTB),  includes  an  empirically  keyed  Biographical  Inventory,  as  well  as  an 
experimental  bank  of  biographical  items. 

The  ASTB  is  a  multiple-choice  test  that  was  developed  and  validated  jointly  by  the 
Naval  Aerospace  and  Operational  Medical  Institute  and  the  Educational  Testing  Services 
to  predict  initial  ground  school  and  primary  flight  training  performance  in  the  Navy, 
Marine  Corps  and  Coast  Guard  pilot  and  naval  flight  officer  (NFO)  curricula.  Ground 
school  includes  coursework  in  basic  engine  properties,  aerodynamics  and  navigation,  as 
well  as  more  advanced  academic  work  on  specific  aircraft  systems  during  the  earliest 
phase  of  their  flight  training.  For  student  naval  aviators,  primary  training  lasts 
approximately  4  months  and  includes  the  above  mentioned  systems  coursework,  flight 
simulator  sessions  and  actual  flight  time  in  the  T-34C  Mentor.  Subsequent  to  their 
completion  of  primary  training,  students  are  placed  into  one  of  3  specific  aircraft  training 
curricula,  the  maritime,  jet,  or  helicopter  pipelines.  The  ASTB  was  not  validated  to 
predict  performance  or  attrition  of  student  naval  aviators  beyond  this  ‘pipelining’. 

The  current  form  of  the  ASTB  test  has  two  equivalent  forms  that  take  approximately  two 
and  one-half  hours  to  administer,  and  includes  six  timed  subtests.  The  subtests  include 
the  Math/Verbal  Test  (MVT),  the  Mechanical  Comprehension  Test  (MCT),  the  Spatial 
Apperception  Test  (SAT),  the  Aviation  and  Nautical  Information  Test  (ANI),  the 
Biographical  Inventory  Test  (BI),  and  the  Aviation  Interest  Test  (AI).  The  MVT  contains 
thirty-seven  questions  and  evaluates  basic  math  skills  and  paragraph  comprehension. 

The  MCT  evaluates  the  candidates’  knowledge  of  basic  mechanical  principles  and 
physics.  The  SAT  measures  a  candidate’s  ability  to  match  an  "inside-out"  or  "cockpit 
view"  to  an  "outside-in"  or  "wingman’s  view".  The  ANI  tests  a  candidate’s  specific 
knowledge  about  aviation  and  nautically-related  material.  The  BI  is  a  71  item  standard 
biographical  inventory  in  which  candidates  respond  to  questions  regarding  their 
academic  background  and  performance,  as  well  as  their  life  experiences.  Factor 
analytical  studies  have  shown  the  71  item  BI  to  measure  traits  such  as  Risk-Taking 
Behavior,  Athletic  Orientation,  Military  Interest,  Engineering  Background  and 
Independence.  The  AI  has  not  been  validated  against  flight  performance  but  was 
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included  in  the  test  battery  to  allow  for  the  continued  validation  of  the  49  alternate  test 
items  that  are  included  in  this  section.  The  goal  was  to  use  these  items  in  future  versions 
of  the  ASTB. 

For  applicants  in  Naval  and  Marine  Corps  aviation  programs,  the  subtest  scores  on  the 
ASTB  are  combined  into  a  total  of  3  composite  scores  for  the  pilot  program,  the 
Academic  Qualification  Rating  (AQR),  the  Pilot  Flight  Aptitude  Rating  (PFAR),  and  the 
Pilot  Biographical  Inventory  (PBI).  The  AQR  and  PFAR  are  weighted  composites  of  the 
MVT,  MCT,  SAT,  and  ANI.  The  AQR  was  validated  to  predict  academic  performance  in 
aviation  preflight  indoctrination  and  ground  school.  The  PFAR  is  validated  against  a 
criterion  of  primary  pilot  performance  (i.e.,  flight  and  simulator  performance).  The  PBI 
score  was  derived  separately  from  the  BI  subtest.  The  PBI  was  validated  to  specifically 
predict  non-medical  student  pilot  attrition  from  ground  school  and  primary  training.  The 
predictive  validity  of  the  ASTB  subtests  for  flight  performance  and  success  in  training 
range  from  .23  to  .40  (Multiple  R),  depending  on  the  subtest  and  specific  criterion  used 
for  analysis  (Frank  &  Baisden,  1992).  The  current  study  is  an  initial  description  and 
evaluation  of  the  items  in  the  AI  subtest  for  possible  inclusion  in  a  revised  version  of  the 
ASTB. 

METHODS 

Sample  Identification 

The  sample  used  for  the  factor  analysis  in  the  current  study  included  all  individuals 
(applicants)  who  took  the  ASTB  between  November  1992  and  January  1998.  A 
subgroup  from  this  sample  (N=  2177)  included  all  individuals  with  completion  data 
(Pass  or  non-medical  Failures)  from  ground  school  and  primary  flight  training  that  had 
been  initially  screened  for  entry  into  flight  training  using  the  ASTB.  This  sample  was 
used  for  the  empirical  keying  and  regression  analysis  of  the  AI  test  items. 

Data  Analysis 

The  Statistical  Package  for  the  Social  Sciences  (SPSS  for  Windows  8.0)  was  the  source 
of  the  statistical  analysis  used  in  the  study.  The  item  factor  analysis  was  done  using 
Principle  Components  Procedures  with  a  Varimax  rotation.  A  factor  analysis  of  the  the 
applicant’s  BI  and  AI  item  responses  was  done  to  evaluate  the  underlying  structure  of 
these  tests.  Using  the  criterion  data  sample,  Chi-square  analysis  was  performed  to 
determine  which  AI  items  were  significantly  related  to  the  pass-fail  criterion.  These 
items  were  then  empirically  keyed  using  a  horizontal  weighting  method  that  was 
identical  to  the  method  used  for  the  original  BI  key  (Morrison,  1991;  Guion,  1965).  The 
SPSS  Logistic  Regression  analysis  program  was  used  to  evaluate  the  incremental 
validity  of  the  AI  test  items  over  the  current  BI  test  score. 
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RESULTS 

A  principle  component  analysis  was  performed  to  assess  the  item  intercorrelations  for 
the  BI  and  AI  tests.  The  principle  component  analysis  and  orthogonal  rotation  yielded  a 
1 1  -component  solution  which  accounted  for  28.1%  of  the  total  item  variance.  Labels  for 
the  principle  components  obtained  were  assigned  based  upon  a  review  of  the  general 
content  of  the  items  that  made  up  the  component.  The  first  component  (Comm-Source) 
was  defined  by  items  reflecting  an  individual’s  commissioning  source  and  accounted  for 
7.9%  of  the  variance.  Commissioning  sources  include  the  Naval  Academy,  ROTC 
programs,  or  Aviation  Officer  Candidate  School.  These  items  all  came  from  the  BI.  The 
second  component  (Military  Orientation)  accounted  for  5.1%  of  the  variance  and 
reflected  an  individual’s  military  orientation  and  interest  prior  to  joining  the  service.  The 
third  component  (Engineering)  accounted  for  2.7%  of  the  item  variance  and  reflected  an 
individual’s  engineering  background  and  classroom  experience.  The  fourth  component 
(Risk-Taking)  accounted  for  2.3%  of  the  variance  and  reflects  an  individual’s 
participation  in  adventurous  activities  and  cited  reasons  for  wanting  to  enter  naval 
aviation.  This  factor  included  a  number  of  AI  test  items.  The  fifth  component  (Science, 
Non-Engineering)  accounts  for  2.1%  of  the  variance  and  reflects  an  individual’s  science 
coursework  in  high  school  and  college.  This  component  primarily  includes  AI  test  items. 
The  sixth  component  (Athletics)  included  both  AI  and  BI  items  reflecting  a  person’s 
athletic  participation  in  high  school  and  college.  This  component  accounted  for  2.0%  of 
the  variance.  The  other  5  factors,  Age,  High  School  Performance,  Swimming 
Experience,  Outdoor  Sports,  and  Rebelliousness/Activism  individually  accounted  for 
less  that  2.0%  of  the  variance.  The  Rebelliousness/Activism  component  was  the  only 
component  to  include  exclusively  AI  test  items.  This  component  reflects  an  individual’s 
past  participation  in  protests  and  their  perception  of  themselves  as  being  rebellious. 

Using  the  training  pass/fail  criterion,  the  AI  test  items  were  evaluated  for  significance 
(p<.05)  using  a  Chi-square  test.  For  each  significant  item,  the  horizontal  weighting 
scoring  method  was  applied  and  a  composite  score  (Alcomp)  for  these  items  was 
developed  for  every  subject.  Out  of  the  total  sample  of  2177,  there  were  a  total  of  189 
non-medical  attritions  from  training.  Of  the  49  total  items  in  the  AI,  38  were 
significantly  related  to  the  criterion  and  included  in  the  AI  key.  Several  of  the  items 
included  in  the  key  were  related  to  the  unique  factor  Rebelliousness/ Activism  in  the  AI 
portion  of  the  test. 

Stepwise  logistic  regression  with  the  pass/fail  criterion  was  done  using  the  Alcomp  and 
the  PBI  score.  The  regression  model  showed  that  the  PBI,  uncorrected  for  range 
restriction,  was  a  significant  predictor  of  the  probability  of  attrition  ( R  =  .096),  however, 
the  Alcomp  score  was  also  a  good  predictor  of  eventual  success  in  the  program  (R  = 
.260). 

DISCUSSION 
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The  results  of  this  study  demonstrate  that  the  Aviation  Interest  subtest  of  the  ASTB  is 
measuring  many  of  the  same  constructs  as  the  current  Biographical  Inventory  that  is 
used  to  screen  applicants  to  the  U.  S.  Navy  pilot  flight  training  curricula.  The  overlap 
between  the  two  tests  was  seen  in  the  engineering  orientation/experiences,  academic 
performance,  and  athletics.  The  component  Rebelliousness/Activism  was  unique  to  the 
AI  subtest,  and  these  items  were  also  found  to  be  predictive  of  success  in  flight  training. 

This  study  also  demonstrated  the  potential  for  a  significant  increase  in  test  validity  using 
the  AI  subtest  in  conjunction  with  the  current  BI  score  for  predicting  success  in  training. 
The  AI  keying,  although  not  cross-validated  using  the  present  sample,  suggests  that  that 
a  substantial  increase  in  predictive  power  can  be  realized  using  these  subtest  items.  Over 
the  last  few  years,  a  number  of  items  from  the  BI  have  been  dropped  due  to  their 
perceived  linkage  to  socio-economic  status  (SES)  in  applicant  populations,  with  a 
corresponding  drop  in  the  predictive  validity  of  the  subtest.  The  incremental  validity 
demonstrated  by  the  AI  items  may  allow  the  substitution  of  these  alternatives  to  maintain 
the  overall  validity  of  the  test  battery,  while  excluding  test  items  which  may  be 
perceived  as  being  culturally  or  SES  sensitive. 
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